System and method for three-dimensional visualization of geographical data

ABSTRACT

A method of using a computer to generate virtual autostereoscopic images from a three-dimensional digital data set is provided. The method includes establishing a first point of view and field of view of a subject volume including a region of interest. The method also includes reading at least one scene parameter associated with the field of view of the subject volume. The method also includes determining a second point of view offset some distance and along some vector from the first point of view based on a value derived from at least one scene parameter. The method also includes generating and storing images and relevant metadata from said first and second points of view. The method also includes displaying the stored images by alternatingly displaying stored images from said first and second points of view. The method further includes performing one or more transformations on the alternatingly displayed images.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/145,719, which entered national stage on Jul. 21, 2011, based onInternational Application No. PCT/US2010/021627, filed on Jan. 21, 2010,which claims the benefit of priority from U.S. Provisional ApplicationNo. 61/146,186, filed on Jan. 21, 2009. The disclosures of all theforegoing applications are herein incorporated by reference in theirentireties.

TECHNICAL FIELD

The presently disclosed embodiments relate to the visualization ofgeographical data and, more particularly, to a system and method forusing parallax information to generate and display autostereoscopic 3Dimages on standard unaided screens.

BACKGROUND

Ever since early humans drew images of their world on cave walls,mankind has endeavored to create images of the environment in which welive. The 6th century Greeks are generally credited with extendingsimple pictures to geographic data visualization with the production ofthe first known maps. The recent advent of geographical Internetbrowsers has carried this concept to an elegant digital extreme. Everycorner of the earth is now detailed and viewable at any time throughpersonal computers or mobile devices. Remote terrain inspections, whichwere the exclusive subject of national security assets only a few yearsago, are now commonplace.

Despite the ability to present and view geographical terrain data fromArkhangelsk to Tierra Del Fuego and all other points around the globe,the vast majority of this data is viewed two-dimensionally. Whilestereoscopic imaging techniques have played an important role in thehistory of modern map making, the necessity of special viewing apparatushas made their adoption in the everyday use of geographical browsersrare. Even where geographical browser providers have includedthree-dimensional terrain data in the landscape model, the views arestill presented to the user two-dimensionally. Every now and then anoccasional enthusiast will go to the trouble of generating stereo pairsfrom three-dimensional terrain data and present them as anaglyph imagesto be viewed using red and blue glasses. Such instances are theexception and not the rule when it comes to viewing geographical data.

The production of two-dimensional images that can be displayed toprovide a three-dimensional illusion has been a long-standing goal inthe visual arts field. Methods and apparatus for producing suchthree-dimensional illusions have to some extent paralleled the increasedunderstanding of the physiology of human depth perception as well asdevelopments in image manipulation through analog/digital signalprocessing and computer imaging software.

Binocular (i.e., stereo) vision requires two eyes that look in the samedirection, with overlapping visual fields. Each eye views a scene from aslightly different angle and focuses it onto the retina, a concavesurface at the back of the eye lined with nerve cells, or neurons. Thetwo-dimensional retinal images from each eye are transmitted along theoptic nerves to the brain's visual cortex, where they are combined in aprocess known as stereopsis, to form a three-dimensional perception ofthe scene being viewed.

Perception of three-dimensional space depends on various kinds ofinformation in the scene being viewed including monocular cues andbinocular cues, for example. Monocular cues include elements such asrelative size, linear perspective, interposition, highlights, andshadows. Binocular cues include retinal disparity, accommodation,convergence, and learned cues including a familiarity with the subjectmatter. While all these factors may contribute to creating a perceptionof three-dimensional space in a scene, retinal disparity may provide oneof the most important sources of information for creating athree-dimensional perception. Particularly, retinal disparity results inparallax information (i.e., an apparent change in the position,direction of motion, or other visual characteristics of an object causedby different observational positions) being supplied to the brain.Because each eye has a different observational position, each eye canprovide a slightly different view of the same scene. The differencesbetween the views represent parallax information that the brain can useto perceive three dimensional aspects of a scene.

A distinction exists between monocular depth cues and parallaxinformation in the visual information received. Both eyes provideessentially the same monocular depth cues, but each eye providesdifferent parallax depth information, a difference that is essential forproducing a true three-dimensional view. Depth information may beperceived, to a certain extent, in a two-dimensional image. For example,monocular depth may be perceived when viewing a still photograph, apainting, standard television and movies, or when looking at a scenewith one eye closed. Monocular depth is perceived without the benefit ofbinocular parallax depth information. Such depth relations areinterpreted by the brain from monocular depth cues such as relativesize, overlapping, perspective, and shading. To interpret monoculardepth information from a two-dimensional image (i.e., using monocularcues to indicate a three-dimensional space on a two-dimensional plane),the viewer is actually reading depth information into the image througha process learned in childhood.

It is known that the act of visual perception is a cognitive exerciseand not merely a stimulus response. In other words, perception is alearned ability which we develop in infancy. Binocular vision is thepreferred method for capturing parallax information by humans andcertain animals. However, other living organisms without the luxury ofsignificant overlapping fields of view have developed other mechanismsto determine spatial relationships.

Certain insects and animals determine relative spatial depth of a sceneby simply moving one eye from side to side. A pigeon bobbing its headback and forth as it walks is a good example of this action. Theoscillating eye movement presents motion parallax depth information overtime. This allows for the determination of depth order by the relativemovement of objects in the scene. Humans also possess the ability toprocess visual parallax information presented over time.

Several mechanical/electronic systems and methods exist for creatingand/or displaying true three dimensional images. These methods may bedivided into two main categories: stereoscopic display methods andautostereoscopic display methods. Stereoscopic techniques includingstereoscopes, polarization, anaglyphic, Pulfrich, and shutteringtechnologies requiring the viewer to wear a special viewing apparatussuch as glasses, for example. Autostereoscopic techniques such asholography, lenticular screens, and parallax barriers produce imageswith a three-dimensional illusion without the use of special glasses,but these methods generally require the use of a special screen.

Certain other systems and methods use square-wave switching and parallaxscanning information to create autostereoscopic displays that allow aviewer to perceive an image as three-dimensional even when viewed on aconventional display. For example, at least one method has beendemonstrated in which a single camera records images while undergoingparallax scanning motion. Thus, the optical axis of a single camera maybe made to move in a repetitive pattern that causes the camera opticalaxis to be offset from a nominal stationary axis. This offset producesparallax information. The motion of the camera optical axis is referredto as parallax scanning motion. As the motion repeats over the pattern,the motion becomes oscillatory. At any particular instant, the motionmay be described in terms of a parallax scan angle.

Over the years the present inventors and their associates have developeda body of work based on methods (optical and synthetic) and apparatusthat capture and display parallax information over time. U.S. Pat. Nos.5,014,126, 4,815,819, 4,966,436, 5,157,484, 5,325,193, 5,444,479,5,699,112, 5,933,664, 5,510,831, 5,678,089, 5,991,551, 6,324,347,6,734,900, 7,162,083, 7,340,094, and 7,463,257 relate to this body ofwork and are hereby incorporated by reference. In addition, U.S. patentapplication Ser. Nos. 10/536,005 and 11/547,714 also relate to this bodyof work and are hereby also incorporated by reference.

To generate an autostereoscopic display based on parallax information,images captured during the scanning motion may be sequentiallydisplayed. These images may be displayed at a view cycle rate of, forexample, about 3 Hz to about 6 Hz. This frequency represents the rate atwhich the parallax information in the sequence is changed. The displayedsequences of parallax images may provide an autostereoscopic displaythat conveys three-dimensional information to a viewer.

Parallax information may also be incorporated into computer generatedimages as described in the aforementioned U.S. Pat. No. 6,324,347 (“the'347 patent”). The '347 patent discloses, inter alia, a method forcomputer generating parallax images using a virtual camera having avirtual lens. The parallax images may be generated by simulating adesired parallax scanning pattern of the lens aperture, and a raytracing algorithm, for example, may be used to produce the images. Theimages may be stored in computer memory on a frame-by-frame basis. Theimages may be retrieved from memory for display on a computer monitor,recorded on video tape for display on a TV screen, and/or recorded onfilm for projection on a screen.

Thus, in the method of the '347 patent, the point of view of a camera(e.g., the lens aperture) is moved to produce the parallax scanninginformation. The ray tracing method of image generation, as may be usedby one embodiment of the method of the '347 patent, may be used togenerate high quality computer images, such as those used in animatedmovies or special effects. Using this ray-tracing method to simulateoptical effects such as depth of field variations, however, may requirelarge amounts of computation and can place a heavy burden on processingresources. Therefore, such a ray tracing method may be impractical forcertain applications, such as 3D computer games, animation, and othergraphics applications, which require quick response.

Another previously mentioned U.S. Pat. No. 7,463,257 (“the '257 patent”)discloses, inter alia, a method for parallax scanning through sceneobject position manipulation. Unlike the moving point of view methodstaught in the '347 patent, the '257 patent teaches a fixed point ofview, and scene objects are moved individually in a coordinated patternto simulate a parallax scan. Even though the final images created usingthe '347 patent and the '257 patent may appear similar, the methods ofgenerating these images are very different.

U.S. patent application Ser. No. 10/536,005 teaches, inter alia, methodsfor critically aligning images with parallax differences forautostereoscopic display. The process requires two or more images of asubject volume with parallax differences and whose visual fields overlapin some portions of each of the images. A first image with an area ofinterest is critically aligned to a second image with the same area ofinterest but with a parallax difference. The images are aligned by meansof a software viewer whereby the areas of interest are criticallyaligned along their translational and rotational axes to converge atsome point. This is accomplished by alternating views of each image atbetween 2 to 60 Hz and adjusting the axial alignment of each imagerelative to one another until a critical alignment convergence isachieved on a sub-pixel level at a point in the area of interest.Autostereoscopic viewing is achieved by alternately displaying (a.k.a.square-wave switching) a repetitive pattern of critically alignedparallax images between 3 and 6 Hz.

Much of the parallax scanning, square-wave switching and other parallaxvisualization prior art deals with capturing, simulating and/orpresenting three-dimensional scenes in which objects and the environmentare reasonably close to the image point of origin (camera sensor).Parallax visualization of geographical data for autostereoscopicthree-dimensional image display on conventional screens, however,presents a different set of circumstances. In general, the square-waveand parallax scanning prior art requires the determination of a point ofconvergence at the time of image capture or computer generation. Thus,these methods are not particularly well suited for parallaxvisualization of imagery generated from large three-dimensional digitaldata sets such as those found in geographical browsers. For example, itis difficult to predetermine and preset a point of convergence whencapturing geographical data for suitable parallax visualization.

The presently disclosed embodiments are directed to overcoming one ormore of the problems associated with prior methods of parallaxvisualization of geographical data. For example, the presently disclosedembodiments may include the capability to capture geographical imageryin an orthographic (parallel viewing) manner. In addition, metadatadescribing the parameters of the captured imagery may also be stored.This allows the stored geographical data to be critically aligned(converged) to multiple points based on the particular requirements ofthe view and/or the display device.

SUMMARY OF THE DISCLOSED EMBODIMENTS

One aspect of the invention includes a method of using a computer togenerate virtual autostereoscopic images from a three-dimensionaldigital data set. The method includes establishing a first point of viewand field of view of a subject volume including a region of interest.The method includes reading at least one scene parameter associated withthe field of view of the subject volume. The method includes determininga second point of view offset some distance and along some vector fromthe first point of view based on a value derived from at least one sceneparameter wherein said second point of view at least partially overlapsthe first field of view and wherein said first and second points of viewcreate a view plane with perpendicular orthogonal views of the subjectvolume. The method also includes generating and storing images andrelevant metadata from said first and second points of view. The methodincludes displaying the stored images by alternatingly displaying storedimages from said first and second points of view at a rate of between 2and 60 cycles per second. Additionally, one or more transformations canbe performed on the alternatingly displayed images that brings somedesired point in the region of interest of both images into a convergedcritical alignment wherein the said region of interest appearsthree-dimensional to a viewer on an standard two-dimensional unaideddisplay.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, represent exemplary features of thedisclosed embodiments and, together with the written description, serveto explain the principles of operation of the disclosed embodiments. Inthe drawings:

FIG. 1 is a flow chart representation of a two-point-of-view displaymethod according to an exemplary disclosed embodiment;

FIG. 2 is a mathematical representation of another two-point-of-viewdisplay method according to an exemplary disclosed embodiment;

FIG. 3 is a diagrammatic representation of a multiple view arrangementaccording to an exemplary disclosed embodiment;

FIG. 4 is a table listing the parallax scan polar angles for each framefor both 24 and 30 frames per second playback; and

FIG. 5 is a diagrammatic representation of parallax scanning points ofview at 4 Hz and 24 frames per second.

DETAILED DESCRIPTION

The present disclosure relates to the parallax visualization ofgeographical data. In this context “geographical data” refers tothree-dimensional digital data sets rendered as terrain models (subjectvolume) available through geographical browsers such as Google Earth andVirtual Earth. For purposes of this disclosure, the term metadata refersto and includes parameters that detail the conditions and settings, andother relevant data, at the time of image capture. Such metadata can bestored along with the image data. For purposes of this disclosure,parallax visualization describes, for example, the capture or generationand presentation over time of parallax data in a manner that will appearthree-dimensional when viewed on a conventional, unaided display.Parallax visualization refers to a form of autostereoscopic display thatexploits certain short-term visual memory and depth mappingpsychophysical visual mechanisms associated with human depth perception.

The advent of freely-available interactive geographical data browserslike Google Earth that contain and present three-dimensional terraininformation makes the additional step of parallax visualizationinfinitely practical. Such browsers are especially attractive forparallax visualization purposes because they have an intuitive userinterface and are easily customized. In addition, the use of XML-basedKeyhole Markup Language (KML) by such browsers for control makescustomization very straightforward through the use of the Network Linktag structure. This allows for the highly accurate remote navigation andvirtual camera control that is required for the subtle adjustments usedto produce realistic depth enhancement through manipulation of parallaxover time.

FIG. 1 provides a flow chart representation of a two-point-of-viewdisplay method according to an exemplary disclosed embodiment. Flowchart 100 outlines first steps in a square-wave application of thepresently disclosed embodiment. Step 101 includes the selection of aregion of interest within the subject volume. Step 102 includes theselection of the direction of view, and step 103 includes the selectionof the field of view. The field of view is typically preset in thebrowser at 60 degrees. The overall view (the area covered in the subjectvolume) is set based on the screen resolution and distance from theregion of interest.

Step 104 includes establishing the first point of view. After a firstpoint of view is determined, which establishes a camera position andview plane, a frame is captured (step 105) and stored (step 106). Alongwith the frame capture, scene parameters can be read (step 107) andstored as relevant metadata from that position.

Next, a second parallel offset point of view is determined andestablished (step 108). The second view is established (step 109) in aposition adjacent to the first point of view perpendicular to the viewplane of the first point of view. The distance of the offset relative tothe first point of view is determined based on a scene parameter suchas, for example, distance to the region of interest. Once the secondpoint of view is established, a frame is captured (step 110) and stored(step 111) with relevant metadata from that position.

Next, the stored frames from the first and second points of view arebrought into critical alignment (step 112) whereby a portion of theregion of interest, which may be all or part of the region, ispositioned so that it occupies the same location in the overlap areas ofboth the first and second points of view. The position of the point ofcritical alignment can be set visually (e.g., manually) (step 113) byalternating the first and second points of view at between 2 to 60 Hzand adjusting the relationship until a critical alignment is achieved.Critical alignment can also be achieved automatically (step 114) usingpredetermined or selected mathematical relationships. In the automatictechnique of step 114, critical alignment may be set based on parameterssuch as, e.g., the distance to the region of interest from the firstpoint of view. Once critical alignment has been achieved, the first andsecond points of view can be displayed autostereoscopically byalternating images at between 3 Hz and 6 Hz (step 115).

FIG. 2 provides a diagrammatic representation of a square-wave displaymethod according to the presently disclosed embodiments. The methodrepresented by FIG. 2 is generally similar to the method outlined by theflow chart of FIG. 1. In the method of FIG. 2, however, a center point11 is selected as a reference for establishing the view of an area ofinterest 33 and for determining the first point of view (POV) 15 and thesecond POV 17. Offset distance S can be determined by a scene parameter(e.g., a distance d_(c)) such that a field of view 20L associated withfirst POV 15 and a field of view 20R associated with second POV 17 fieldof view 20R overlap area of interest 33 in a desired amount. Once POV 15and POV 17 are established, images may be captured and stored withrelevant metadata from the respective positions. The diagram of FIG. 2outlines the information necessary and, therefore, the procedure forauto-converging a pair of images rendered from offset positions withrespect to center point 11 and corresponding to first POV 15 and secondPOV 17, respectively.

Offset distance S is a significant parameter to the methods of thepresently disclosed embodiments. In determining offset distance S, it ishelpful to develop a process that works well with an individual user'sdisplay screen and geographical browser. Certain exercises are helpfulin putting the various parameters in context. For example, if we use thehuman visual model as a basis for determining an angle of view it wouldbe two eyes with a separation of 65 mm viewing an object at one meterdistance creating an angle of view for each eye of approximately 1.86degrees. This can be mathematically described as follows:separation=65 mm*1 m/1000 mm=0.065 mhalf (for each eye) separation=0.065 m/2=0.0325 mhalf angle (angle of convergence)=arctan(0.0325 m/1 m)=1.8614576 degrees

If the human model is applied to geographical data (e.g., viewing anarea of interest from a distance of, for example, one thousand meters),the method represented in FIG. 2 it would yield exemplary parameters, asfollows:half angle=1.8614576 degrees=arctan((half separation)/1000 m)half separation=1000 m*tan(1.8614576 degrees)=32.5 metersseparation=half separation*2=65 meters

The previous exercise is not directly applicable when it comes topresenting parallax information over time. Alternating images between 3to 6 Hz with parallax differences of 1.86 degrees produces images thatcan be unstable and difficult to view. However, the exercise is notfutile when it comes to understanding the human visual mechanics. Theperception of parallax information presented over time can beaccomplished using small angles. For this reason it may be useful totest the particular geographical browser to determine what works wellwith the parallax visualization method being applied at a particularscreen resolution. As a rule of thumb a scaling factor can be applied tothe human model of 1.86 degrees (i.e., the half angle) on the order of 2to 15 times smaller.

With respect to FIG. 2, applying a 1000 meter distance, d_(c), fromcenter point 11 to area of interest 33 and applying a scaling factor of13 to the human model angle of 1.86 degrees with a field of view of 60degrees for display on a screen with a horizontal resolution of 1280pixels results in the following:half angle=1.86/13=0.1431 degreeshalf separation=S=1000 m*tan(0.1431)=2.497 meterspixel shift for convergence=X=E*S/d _(c) =E*tan(half angle)

-   -   where the distance from the camera to the view plane in pixels,    -   E=half horizontal resolution/tan(half horizontal field of view)    -   horizontal resolution=hpix=1280 pixels    -   horizontal field of view=hFOV=60 degrees

Under these conditions, E=640 pixels/tan(30 degrees)=369.5 pixels, andX=369.5*2.497/1000=369.5*0.002497=0.923 pixels.

FIG. 2 also leads to the procedure for auto-converging a pair of storedimages, corresponding to the first POV 15 and the second POV 17,respectively, rendered by a geographical browser such as Google Earth.After choosing X (i.e., the number of pixels to shift POV 15 or POV 17images to converge), with hFOV and hpix giving E, and d_(c) given by thebrowser as (camera altitude—terrain altitude, in meters), we have:d_(L)=d_(R)=d_(C).

In an exemplary application, the method represented by FIG. 2, asapplied to a geographical browser like Google Earth, would progress asfollows:

-   -   Supply center point 11 to Google Earth;    -   Google Earth responds with scene parameters including horizontal        field of view angle, hFOV, horizontal pixel extent of view,        hpix, and distance to terrain centered in the given view, d_(T).    -   from these scene parameters, a distance E to the view plane is        calculated as E=hpix/2/tan(hFOV/2) in pixels    -   choose a distance to convergence, d_(C), usually <=d_(T), and a        pixel offset, x, to calculate the offset distance, S=x*d_(C)/E        in meters    -   supply a first offset point of view, POV 15, to Google Earth by        shifting the center point 11 by S meters to the left,        perpendicular to the original view    -   capture and store an image of this first point of view along        with relevant metadata    -   supply a second offset point of view, POV 17, to Google Earth by        shifting the center point 11 by S meters to the right,        perpendicular to the original view    -   capture and store an image of this second point of view along        with relevant metadata    -   critically align the stored images by shifting the right image        2*x to the left, or by shifting the left image 2*x to the right,        or by shifting each image by x in the respective opposite        direction.

Critical alignment by more than x is equivalent to having chosen asmaller distance to convergence. Critical alignment by less than x isequivalent to having chosen a larger distance to convergence, assumingthe same value of the offset distance, S.

A further method according to the presently disclosed embodiments isrepresented by FIG. 3. While the method of FIG. 2 includes the captureof views for square-wave display, the method of FIG. 3 includes thearrangement of multiple points of view (e.g., more than two points ofview) offset from center point 11 and captured for display as parallaxscanned images.

In the method of FIG. 3, center point 11 is selected as a reference forestablishing the view of area of interest 33 and also for determiningthe polar coordinates for the placement of the parallax scan points ofview. Each point of view is determined, for example, by the separation(radius), the scan frequency, the current frame count, and the framerate. The geometry of the scan path is typically elliptical or circularbut can also include other geometries depending on the requirements of aparticular application. The scan path may be random, algorithmic, oreven determined by some external function like a sound source. Assumingthe parallax scan path is a perfect circle, each successive point ofview will have a constant angular separation defined as:360 degrees/cycle*scan frequency (cycles/second)/frame rate(frames/second)For example, a scan frequency of 4.4 Hz and a frame rate of 30 framesper second gives an angular separation of 360*4.4/30=52.8 degrees.

The progression of the polar coordinates of parallax scan points of viewmay be accomplished by assigning the first parallax scan position to thechosen initial angle and radius. Subsequent positions may be determinedby adding a constant separation angle to the current position whilemaintaining a constant radius (or half separation). The polarcoordinates for a particular frame in a sequence may be defined, forexample, as:(frame number*separation angle+initial angle,radius)Each frame represents the angular component of the polar coordinates fora parallax scan position. Here, the frame number counts from 0, theinitial angle is determined by the situation or the requirements of theuser, the separation angle is as defined as above, and the radius is thesame as the half separation defined earlier. These polar coordinates mayalso be expressed as a Cartesian coordinate vector. For example, given aset of polar coordinates (angle, radius), and, assuming clockwiserotation with an angle of 0 aligned with the y-axis, one can calculateCartesian coordinates (x,y) as (radius*sin(angle), radius*cos(angle)). Atable illustrating the parallax scan polar angles for each frame forboth 24 and 30 frames per second playback is provided in FIG. 4.

In the method represented by FIG. 3, the offset distance S for eachparallax scan position can be determined by a scene parameter, such as adistance d_(c), in a manner similar to the square-wave methods discussedabove. Each parallax scan position may be offset by value S from centerpoint 11 and located in its proper position along the parallax scanpath. As shown in FIG. 3, POV 15 is located at a distance S from centerpoint 11 and at a parallax scanning position of 270 degrees. Similarly,POV 17 is located at a distance S from center point 11 and at a parallaxscanning position of 90 degrees. Multiple parallax scanning positionsmay be selected and used from along the 360 degree parallax scanningpath. FIG. 4 provides an exemplary table of such multiple parallaxscanning positions corresponding to frame rates of both 24 frames persecond and 30 frames per second.

Applying the method of FIG. 3, FIG. 5 illustrates the polar positions ofa 4 Hz parallax scan at 24 frames per second. Shown in FIG. 5 is thecenter point 11 along with six points of view (POV 1 . . . POV 6). Eachpoint of view is offset from the center point 11 by an offset distanceS. FIG. 5 also illustrates a view plane 504 and multiple orthogonal view506. The area being imaged includes a region of interest 33 selectedfrom the subject volume 510. The center point 11 is located at anorthogonal distance d_(C) from the region of interest 33.

The presently disclosed embodiments include techniques for locating andcapturing orthographic (parallel) views for square-wave and parallaxscanning autostereoscopic display of geographical data from a fixedposition. The presently disclosed embodiments methods can also beapplied to capturing data from positions along a motion path forpresentation as moving images (movie).

Building such a movie can be accomplished through a number of steps,including:

-   -   defining a path through a chosen scene    -   instructing the geographical browser to step along the path    -   calculating parallax view positions    -   capturing and storing frames    -   critically aligning frames    -   assembling frames as an animation        These steps are further described below using Google Earth as an        exemplary platform.

The step of defining a path through a chosen scene first includes usingGoogle Earth's Path tool to specify a sequence of points following asection of landscape. The KML data describing the path can then beexported to a text file containing a comma-delimited list of points(longitude, latitude, elevation, and heading). It may also be suitableto acquire a sequence of points algorithmically or from some othergeographic data source.

To aid in path definition, software can be provided to performinterpolation of the points along the path to provide smooth transit. Asimple linear interpolation can be used initially to simplify thecalculations for constant distance steps along the selected path and tosmooth the changes in heading. In addition, the calculations can beexpanded to use a cubic spline and the Composite Simpson's Rule to solvefor path length. The tangent at each constant-distance step can be usedfor the heading. The smoothing process requires a conversion from thecoordinates of Google Earth (spherical or ellipsoidal) to Cartesiancoordinates and back. Google Earth uses what is called the SimpleCylindrical projection of the WGS84 datum. Usually, the SimpleCylindrical projection is only used with a sphere so a direct conversioncan be provided. A customized interface can be used with the UniversalTransverse Mercator projection of the standard ellipsoid to calculatevery accurate approximations over the short distances being used.

The smoothed data points can be fed back into Google Earth to acquireterrain elevations at each point. Google Earth's terrain model can beused to obtain elevations and to set limits on the elevations that arepresented back to the program for the final visualization step. Suchelevation limits can aid in minimizing or eliminatinginterface-generated errors that may result from elevations located belowthe surface of a terrain model for the associated position. Appropriateelevations facilitate production of a smooth, “crash-free” flight path.The elevation profile could also be acquired from an external terrainmodel, as an extension.

In response to instructions for Google Earth to step along the definedpath, the Google Earth network interface will pass back a complete setof scene parameters, including elevation for each position it receives,allowing the user to accumulate a profile for the chosen path. At thispoint the opportunity exists to smooth and meld the path by modifyingthe tilt (pitch: nose up and down) and the roll (rotation along the axisof motion to make turns more realistic), and to smooth the elevations.

Once a smooth path is obtained, the flight plan can be visualized.Google Earth's network interface provides the user a completeprogrammatic control over the virtual camera. Each call from the networkinterface can be answered with the data corresponding to the next frameto be captured, and each step can be provided with an appropriate offsetto produce parallax in the rendered scene.

Next, the parallax view positions can be calculated. The primary effectfrom parallax depth-enhancement comes from the choice of the offset orview separation. A smaller view separation corresponds to a convergencepoint (apparent scene depth where there is no visible pixel motion fromone frame to the next) that is closer to the camera, while a larger viewseparation corresponds to a convergence point that is farther from thecamera. This is the inverse of the pixel separation, which is the numberof pixels to shift each image to critically align them at the chosenconvergence point. A smaller pixel separation corresponds to aconvergence point that is farther from the camera, while a larger pixelseparation corresponds to a convergence point that is closer to thecamera.

There are several strategies that can be used for deciding where onewants the convergence to be in a series of images. One can use a fixeddepth so that the convergence never changes from frame to frame. One canuse an adaptive method, which tracks the objects in a region of visualinterest, and can choose to converge on or near those objects. Or, onecan choose to converge optimally for an entire scene by finding therange of depths of objects and empirically finding the “sweet spot” thatbrings the scene to life.

In general, though, the process relies most heavily on the calculationof the offset of the camera from its previously determined path toproduce the desired parallax. Google Earth provides the virtual camerawhich is defined by its position (longitude, latitude, altitude),orientation (heading, tilt, roll), and its view parameters (horizontaland vertical pixel resolution, horizontal and vertical field of view,and view plane distance (also known as eye distance, or, indirectly,focal length)). Google Earth also provides the fields of view and theresolution of the display window from which the distance to the viewplane in pixels may be calculated. For example:E=(horizontal resolution)/(2*tan(horizontal field of view/2))

At this point, the user can either choose the pixel separation or theview separation (in meters) for a given distance (in meters) from theeye (or camera) for convergence. A constant distance may be used, thedistance from the camera to the terrain for each point (variable), oreven a smoothed version of the latter.

Given the pixel separation, the view separation can be represented as:S=((pixel separation)*(distance to convergence))/(view plane distance)Or, given the view separation, the pixel separation can be representedas:

$X = \frac{\left( {{view}\mspace{14mu}{separation}} \right)*\left( {{view}\mspace{14mu}{plane}\mspace{14mu}{distance}} \right)}{\left( {{distance}\mspace{14mu}{to}\mspace{14mu}{convergence}} \right)}$

Again, the view separation is the distance from the camera position onthe path to the offset camera position, while the pixel separation isthe distance in the view plane that each image must be moved to alignthe converged distance. With these values known, the new camera positioncan be calculated from the original camera position along the path. Thisis very simple for the stereo square-wave case where the camera isoffset perpendicular to the heading (east or west, left or right) to getpositions of parallax. For the parallax scanning case, the inverse viewtransformation may be calculated for the desired angular displacement onthe circle in the plane of the camera. This may be accomplished using aset of matrix multiplications that mimic steps from the renderingpipeline of OpenGL and many other 3D computer graphics renderingenvironments, where the order of matrix operations can proceed asfollows:[the position in space of the offset camera]=[the position in space ofthe original camera]*[the rotation about the y-axis for theheading]*[the rotation about the x-axis for the tilt]*[the rotationabout the z-axis for the roll]*[the rotation about the z-axis for thecamera's angular displacement in the scan circle]*[the translation alongthe y-axis for the view separation]

Next, the frames can be captured and stored based on the defined camerapositions from which Google Earth will render. This information is sentalong with the associated orientation parameters to Google Earth inresponse to a network interface call. It is assumed that Google Earthwould render the next frame from this point of view. The program's nextcall contains the location and orientation of its most-recently renderedframe, which closely matches the parameters that were sent. Thistriggers a call to an operating system-dependent screen capture utilitythat saves the current frame in a sequentially numbered file. Theprocess continues for each smoothed point. Upon completion of thisprocess, a set of frames will be available that can be accumulated intoan animated movie for presentation and analysis.

The captured set of frames can then be critically aligned. In generatingthe position and orientation data for each view (frame), criticalalignment parameters are also generated. After each frame is captured itis aligned to match the selected and calculated values to achieve thedesired parallax scan. This is accomplished by shifting an image by thehorizontal and vertical pixel amount calculated by the process. Thefollowing exemplary parameters illustrate this process in action:

-   -   convergence distance: 1000 m (given)    -   separation: 2.5 m (given)    -   angle of parallax: arctan(2.5/1000)=0.1432 degrees    -   scan frequency: 4.4 Hz (given)    -   frame rate: 30 frames per second (given)    -   separation angle: 360*4.4/30=52.8 degrees    -   initial angle: 0 degrees (given)    -   frame number: 42 (counting from 0) (chosen for example)    -   polar coordinates of scan position:

$\begin{matrix}{= \left( {{{{frame}\mspace{14mu}{number}*{separation}\mspace{14mu}{angle}} + {{initial}\mspace{14mu}{angle}}},{radius}} \right)} \\{= {\left( {{{42*52.8} + 0},2.5} \right) = {\left( {2217.6,2.5} \right) = \left( {57.6,2,5} \right)}}}\end{matrix}$

-   -   Cartesian coordinates (S vector):        -   =(2.5*sin(57.6), 2.5*cos(57.6))=(2.111, 1.340)    -   horizontal field of view: 60 degrees (supplied by Google Earth)    -   horizontal pixel resolution: 1280 pixels (supplied by Google        Earth)    -   E (view plane distance)=hpix/2/tan(hFOV/2)=1108.5125168440818    -   x=E*SMC=E*2.5*sin(57.6)/1000=2.340 pixels to shift horizontally    -   y=E*SY/dC=E*2.5*cos(57.6)/1000=1.485 pixels to shift vertically

The direction of the shift for any given frame is usually the negativeor opposite of the sign indicated by the calculations. Typically, thecoordinate system in the view plane has the positive direction as rightor up for x and y respectively while the coordinate system in the imagespace has the positive direction as right and down for x and yrespectively. Also, a positive offset in the view plane corresponds to anegative shift (in view plane coordinates) to achieve alignment. So, inthe above calculations, both components of the final separation vectorare positive which means that the horizontal pixel shift should benegative (to accommodate the alignment direction) while the verticalpixel shift should be positive (to accommodate the alignment directionand the image coordinate system direction).

Next, the aligned frames can be assembled together into an animatedmovie. This movie will constitute a depth-enhanced version of afly-through of the selected region of interest.

It should be noted that the methods of the presently disclosedembodiments, as described above, may be accomplished using any suitablecomputing device. For example, any of today's modern computers can beconfigured with appropriate software for executing the computational anddisplay techniques described above. These methods may also beaccomplished as part of a pre-processing or predetermined processingroutine the results of which may be configured for later display on auser screen. Alternatively, or additionally, the described methods ofgenerating and critically aligning images according to any of themethods described above may be accomplished in real-time or in nearreal-time by the viewer. In such real-time or near real-time processes,computers may be employed that calculate and display the above-describedcritically aligned images, e.g., as input is received (or withinmilliseconds of receiving input) from a user or as input, in the form ofcomputational output generated by a processor, becomes available forcontinued processing.

Additional advantages and modifications will readily occur to thoseskilled in the art. The invention in its broader aspects is, therefore,not limited to the specific details, representative algorithms andillustrative examples shown and described. Accordingly, departures maybe made from such details without departing from the spirit or scope ofapplicants' inventive concept.

What is claimed is:
 1. A method of using a computer to generate virtualautostereoscopic images from a three-dimensional digital data set,comprising: establishing a first point of view comprising a first fieldof view of a subject volume including a region of interest; reading ascene parameter associated with the first field of view of the subjectvolume; determining a second point of view offset some distance andalong some vector from the first point of view based on a value derivedfrom the scene parameter wherein said second point of view comprises asecond field of view that at least partially overlaps the first field ofview and wherein said first and second points of view create a viewplane perpendicular to orthogonal views of the subject volume;generating and storing images and relevant metadata from said first andsecond points of view; determining a critical alignment transformationfor at least one stored image based on the offset distance and an angleof convergence of at least one of the first point of view and the secondpoint of view; performing a transformation using the critical alignmenttransformation on the at least one stored image that brings some desiredpoint in the region of interest of the at least one stored image into aconverged critical alignment wherein the region of interest appearsthree-dimensional to a viewer on a standard two-dimensional unaideddisplay; and displaying the stored images by alternatingly displayingthe transformed images from said first and second points of view at arate of between 2 and 60 cycles per second.
 2. The method of claim 1,wherein said first point of view is established based on a central pointof view of the region of interest in the subject volume, and wherein anadditional point of view is offset, with respect to at least one of thecenter point of view or the first point of view, by some distance andalong some vector in the view plane with orthogonal overlapping fieldsof view of the region of interest, wherein the determination and offsetof the additional point of view is based on the scene parameterassociated with the subject volume.
 3. The method of claim 2, whereinsaid additional point of view is arranged at polar coordinates in theview plane that includes the center point of view, the first point ofview, and the second point of view.
 4. The method of claim 2, furthercomprising: progressing the first point of view step-by-step along asimulated flight path; and determining the additional point of view inthe view plane on a frame-by-frame basis at a progressing polarcoordinate offset some distance along a vector at each step of theflight path where an additional image is generated and stored.
 5. Themethod of claim 1, wherein the scene parameter being read is a distancefrom a center point of view to a point in the region of interest.
 6. Themethod of claim 4, further comprising adjusting the additional point ofview, offset distances, and vectors on a frame-by-frame basis based onchanges in one or more scene parameters.
 7. The method of claim 1,wherein displaying the stored images by alternatingly displaying thetransformed images from said first and second points of view at a rateof between 2 and 60 cycles per second comprises displaying the storedimages in a square-wave switching manner at a rate of between 3 and 6cycles per second.
 8. The method of claim 3, further comprising:generating and storing images from the additional point of view; anddisplaying the images generated and stored from the additional point ofview in a parallax-scanning manner.
 9. The method of claim 1, whereinthe transformation comprises an orthogonal image transformation that isbased on at least one of scene or view screen resolution parameters. 10.The method of claim 1, wherein the images from the first and secondpoints of view are generated and critically aligned in in nearreal-time.
 11. The method of claim 4, further comprising assembling theimages generated and stored through progression of the simulated flightpath for display as a parallax scanning movie.
 12. The method of claim1, further comprising assembling the images generated from the first andsecond points of view for display as a square-wave switching movie. 13.The method of claim 1, further comprising: progressing the first pointof view step-by-step along a simulated flight path; determining theadditional point of view in the view plane on a frame-by-frame basis ata square-wave switching offset some distance along a vector at each stepof the flight path; and generating and storing an image at each step ofthe flight path.
 14. The method of claim 13, further comprisingassembling the images generated and stored through progression of thesimulated flight path for display as a movie.
 15. A computer system,comprising: a display; a memory; and a processor coupled to the displayand memory, and configured with processor-executable instructions toperform operations comprising: establishing a first point of viewcomprising a first field of view of a subject volume including a regionof interest; reading a scene parameter associated with the first fieldof view of the subject volume; determining a second point of view offsetsome distance and along some vector from the first point of view basedon a value derived from the scene parameter wherein said second point ofview comprises a second field of view that at least partially overlapsthe first field of view and wherein said first and second points of viewcreate a view plane perpendicular to orthogonal views of the subjectvolume; generating and storing images and relevant metadata from saidfirst and second points of view; determining a critical alignmenttransformation for at least one of the stored images based on the offsetdistance and an angle of convergence of at least one of the first pointof view and the second point of view; performing a transformation usingthe critical alignment transformation on the at least one stored imagethat brings some desired point in the region of interest of the at leastone stored image into a converged critical alignment wherein the regionof interest appears three-dimensional to a viewer on a standardtwo-dimensional unaided display; and displaying the stored images byalternatingly displaying the transformed images from said first andsecond points of view at a rate of between 2 and 60 cycles per second.16. The computer system of claim 15, wherein said first point of view isestablished based on a central point of view of the region of interestin the subject volume, and wherein an additional point of view isoffset, with respect to at least one of the center point of view or thefirst point of view, by some distance and along some vector in the viewplane with orthogonal overlapping fields of view of the region ofinterest, wherein the determination and offset of the additional pointof view is based on the scene parameter associated with the subjectvolume.
 17. The computer system of claim 16, wherein the additionalpoint of view is arranged at polar coordinates in the view plane thatincludes the center point of view, the first point of view, and thesecond point of view.
 18. The computer system of claim 16, furthercomprising: progressing the first point of view step-by-step along asimulated flight path; and determining the additional point of view inthe view plane on a frame-by-frame basis at a progressing polarcoordinate offset some distance along a vector at each step of theflight path where an additional image is generated and stored.
 19. Thecomputer system of claim 15, wherein the scene parameter being read is adistance from a center point of view to a point in the region ofinterest.